Minimum Degree Reordering Algorithms: A Tutorial

The problem of matrix inversion is central to many applications of Numerical Linear Algebra. When the matrix to invert is dense, little can be done to avoid the costly O(n) process of Gaussian Elimination. When the matrix is symmetric, one can use the Cholesky Factorization to reduce the work of inversion (still O(n), but with a smaller coefficient). When the matrix is both sparse and symmetric, we have even more options. An entire universe of approximation algorithms exists to take advantage of the structure of sparse symmetric matrices, but some applications still require computing the “true” inverse. In these cases we must once again fall back on Cholesky, but now we use a variant called Sparse Cholesky and the amount of work required to invert the matrix has changed. Slight alteration in the ordering of our equations and unknowns in our sparse, symmetric problem yields vastly different Sparse Cholesky runtimes. Why? This is the phenomenon of fill-in. Figure 1 shows the process at work. Each step of Cholesky eliminates a row and a column from the matrix. Nonzero elements of our row a above the diagonal lead to new nonzero elements in those rows which have nonzeros in the column a. These new nonzeros are the so called fill-in and they spell trouble for the Sparse Cholesky algorithm in the form of more floating point operations. A remarkable phenomenon is that fill-in depends on the order in which we position the elements of the matrix. Figure 2 shows the same matrix as figure 1 but with the first and last columns and rows interchanged. This second configuration creates no fill in and Sparse Cholesky finishes faster. The task now seems obvious: find the ordering of the columns and rows of the matrix that creates the least fill-in. If we could do this, then we could minimize the work to invert any sparse, symmetric matrix by reordering using permutation matrices. Bad news arrives from the world of graph theory but to understand it, we must recast the fill-in process in the language of graphs. Any symmetric matrix corresponds to an undirected graph called the elimination graph (see figure 3). To build the elimination graph, create a vertex for every row and then for every row a that has a nonzero above the diagonal in the column b, construct an edge between the vertices a and b. The act of eliminating a row (and creating the resulting fill-in) using Cholesky corresponds to removing a vertex a and its corresponding edges, then forming a clique from the former neighbors of a. The fill-in corresponds